Interpolating Conditional Density Trees

Joint distributions over many variables are frequently modeled by decomposing them into products of simpler, lower-dimensional conditional distributions, such as in sparsely connected Bayesian networks. However, au­ tomatically learning such models can be very computationally expensive when there are many datapoints and many continuous vari­ ables with complex nonlinear relationships, particularly when no good ways of decom­ posing the joint distribution are known a pri­ ori. In such situations, previous research has generally focused on the use of discretization techniques in which each continuous vari­ able has a single discretization that is used throughout the entire network. In this paper, we present and compare a wide variety of tree-based algorithms for learning and evaluating conditional density estimates over continuous variables. These trees can be thought of as discretizations that vary ac­ cording to the particular interactions being modeled; however, the density within a given leaf of the tree need not be assumed con­ stant, and we show that such nonuniform leaf densities lead to more accurate density esti­ mation. We have developed Bayesian net­ work structure-learning algorithms that em­ ploy these tree-based conditional density rep­ resentations, and we show that they can be used to practically learn complex joint prob­ ability models over dozens of continuous vari­ ables from thousands of data points. We focus on finding models that are simultaneously ac­ curate, fast to learn, and fast to evaluate once they are learned.